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Course Aims 


m understanding of computability, complexity and 
intractability; 


m knowledge of lambda calculus, types, and type safety. 
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Course Outcomes 


By the end of the course you should be able to 


m Explain (non-)deterministic finite and pushdown automata 
and use the pumping lemma to show languages 
non-regular 


m Explain decidability, undecidability and the halting 
problem. 


m Demonstrate the use of reductions for undecidability 
proofs. 


m Explain the notions of P, NP, NP-complete. 
m Use reductions to show problems to be NP-hard. 
m Write short programs in lambda-calculus. 
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Cours 


Motivation 
o0 


e Outline 


Introduction. Finite automata. 

Regular languages and expressions. 
Context-free languages and pushdown automata. 
Register machines and their programming. 
Universal machines and the halting problem. 
Decision problems and reductions. 
Undecidability and semi-decidability. 
Complexity of algorithms and problems. 

The class P 

Non-determinism and NP 

NP-completeness 

Beyond NP. 

Lambda-calculus. 

Recursion. 

Types. 
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Assessment 


The course is assessed by a written examination (80%) and 
two coursework exercises, the first formative and the second 
summative (for the remaining 20%). 

Coursework deadlines: End of weeks 5 and 9. 
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Textbooks 


It will be useful, but not absolutely necessary, to have access 
to: 


= Michael Sipser Introduction to the Theory of Computation, 
PWS Publishing (International Thomson Publishing) 


m Benjamin C. Pierce Types and Programming Languages, 
MIT Press 


There is also much information on the Web, and in particular 
Wikipedia articles are generally fairly good in this area. 
Generally I will refer to textbooks for the detail of material I 
discuss on slides. 
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What is computation? What are computers? 


Some computing devices: 
The abacus — some millennia BP. 


[Association pour le musée international du calcul de l’informatique 
et de l’automatique de Valbonne Sophia Antipolis (AMISA)] 
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First mechanical digital calculator — 1642 Pascal 


—$— 


[original source unknown] 
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The Difference Engine, [The Analytical Engine] - 1812, 1832 
Babbage / Lovelace. 


[ Science Museum ?? ] 


Analytical Engine (never built) anticipated many modern 
aspects of computers. See 
http://www.fourmilab.ch/babbage/. 
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ENIAC — 1945, Eckert & Mauchle 
ee IT ae 


[University of Pennsylvania] 
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What do computers manipulate? 


Symbols? Numbers? Bits? Does it matter? 
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What do computers manipulate? 


Symbols? Numbers? Bits? Does it matter? 

What about real numbers? Physical quantities? Proofs? 
Emotions? 

Do we buy that numbers are enough? If we buy that, are bits 
enough? 

How much memory do we need? 
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What can we compute? 


If we can cast a problem in terms that our computers 
manipulate, can we solve it? 
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What can we compute? 


If we can cast a problem in terms that our computers 
manipulate, can we solve it? Always? Sometimes? With how 
much time? With how much memory? 


We will seek mathematical answers to these questions. For 
that, we will need a model of computation. 


Finite Automata 
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Finite Automata 


A finite automaton takes 
a string as input and says 
“yes” or now 


Define the language of a fi- 
nite automaton A, written, 
L(A) to be the set of strings 
for which A says “yes”. 


A string is a (possibly-empty) 
sequence of symbols from a 
set called an alphabet, usu- 
ally written È. 
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DFAs, formally 


A deterministic finite automaton (DFA) is a quintuple 
(Q, È, qo, 5, F) where: 


m Q isa finite set of states, 


Finite Automata 
0000000000000 


DFAs, formally 


A deterministic finite automaton (DFA) is a quintuple 
(Q, È, qo, ©, F) where: 

m Q is a finite set of states, 

m Dis the alphabet, the set of symbols, 


Finite Automata 
0000000000000 


DFAs, formally 


A deterministic finite automaton (DFA) is a quintuple 
(Q, È, qo, ©, F) where: 

m Q is a finite set of states, 

m Dis the alphabet, the set of symbols, 

m go € Q is the initial state 


Finite Automata 
0000000000000 


DFAs, formally 


A deterministic finite automaton (DFA) is a quintuple 
(Q, £, qo, ©, F) where: 

m Q is a finite set of states, 

m Dis the alphabet, the set of symbols, 

m go € Q is the initial state 

m ò: Q xE -— Qis the transition function, 


Finite Automata 
0000000000000 


DFAs, formally 


A deterministic finite automaton (DFA) is a quintuple 
(Q, £, qo, ©, F) where: 

m Q is a finite set of states, 

m Dis the alphabet, the set of symbols, 

m go € Q is the initial state 

m ô: Q xE -— Qis the transition function, 

m F C Qis the set of final states. 


Finite Automata 
0600000000000 


DFAs, formally 


A deterministic finite automaton (DFA) is a quintuple 
(Q, È, qo, ©, F) where: 

m Q is a finite set of states, 

m Dis the alphabet, the set of symbols, 

m dE Q is the initial state 

m ò: Q xE -— Qis the transition function, 

m F C Qis the set of final states. 


Exercise: What is the formal definition of our example? 
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Languages 


A DFA accepts a string w € X* iff ò“ (qo, w) € F, where 5* is ò 
applied successively for each symbol in w. The language of a 


DFA £(A) C X* is the set of all strings accepted by A. 
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Determinism 


In a DFA, the transition function is a total function which gives 
exactly one next state for each input symbol (it’s deterministic). 


Questions 


Does relaxing any of these requirements affect the set of 
languages we can recognise? How would we prove this? 
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Determinism 


In a DFA, the transition function is a total function which gives 
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Nondeterministic Finite Automata 


A nondeterministic finite automaton (NFA) is a quintuple 
(Q, £, qo, ò, F) where: 

m Q is a finite set of states, 

m Dis the alphabet, the set of symbols, 

m go € Q is the initial state 

m ô: Q xE — P(Q) is the transition function, 

m F C Qis the set of final states. 
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Nondeterministic Finite Automata 


A nondeterministic finite automaton (NFA) is a quintuple 
(Q, £, qo, 5, F) where: 

m Q is a finite set of states, 

m Dis the alphabet, the set of symbols, 

m go € Q is the initial state 

m ô: Q xE — P(Q) is the transition function, 

m F C Qis the set of final states. 


Note the only difference here is the transition function, which 
gives a set of next states for a given symbol. 
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Nondeterministic Finite Automata 


Arun of an NFA A on a string w = a1a2 .. . a, is a sequence of 


states qoqi ...q, in Q such that: 
m qo is the initial state 


m forall; = 1...k we have q; € 5(qj_1, a;). 
A run is accepting if the last state q, € F. 
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Nondeterministic Finite Automata 


A run of an NFA A on a string w = a1a2 .. . aç is a sequence of 
states qoqi -- -qk in Q such that: 

m qo is the initial state 

m forall; = 1...k we have q; € 8(qj_1, a). 
Arun is accepting if the last state q, € F. 


The nondeterminism means that we have multiple alternative 
computations. For our purposes we will use angelic 
non-determinism, which says that we achieve success if any of 
our alternatives succeed. 


L(A) = {w | there exists an accepting run of A on w} 
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A run of an NFA A on a string w = a1a2 .. . aç is a sequence of 
states qoqi -- -qk in Q such that: 


m qo is the initial state 
m forall; = 1...k we have q; € 8(qj_1, a). 
Arun is accepting if the last state q, € F. 


The nondeterminism means that we have multiple alternative 
computations. For our purposes we will use angelic 
non-determinism, which says that we achieve success if any of 
our alternatives succeed. 


L(A) = {w | there exists an accepting run of A on w} 


Exercise: Is 10100 in the language of our previous example? 
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NFA = DFA 


Making finite automata non-deterministic does not change 
their expressivity. That is, for every non-deterministic 
automaton A there is a deterministic automaton D such that 
£(D) = L(A) and vice versa. 
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NFA = DFA 


Making finite automata non-deterministic does not change 
their expressivity. That is, for every non-deterministic 
automaton A there is a deterministic automaton D such that 
£(D) = L(A) and vice versa. 


m DFA > NFA:Easy, the DFA is already an NFA where the 
transition function always returns a singleton set. 


m NFA > DFA:We will use the subset construction. 
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Subset Construction 


Key Idea 


For an NFA A, the corresponding DFA D tracks the set of states 
that A could possibly be in, given the string read so far. So, 
each state of D is a set of states from A. 


Example (From earlier) 
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Formally 


The Subset Construction 


Given an NFA (Q4, , qo, õa, FA), construct a DFA 
(P(Qa).E, {q0}, 8p, Fp) where: 


8p(S,a) = LJ 8a(g,a) foreach Sc Q 
ges 


and 
Fp=1o 6 Ol| SAFA 
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Formally 


The Subset Construction 


Given an NFA (Q4, , qo, õa, FA), construct a DFA 
(P(Qa), £, {q0}, 8p, Fp) where: 


8p(S,a) = LJ 8a(g,a) foreach Sc Q 
qES 


and 
Fp = {S GO SAFA) 


Question: If our NFA has n states, how many states could our 
DFA have? 
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Proving that this is correct (i.e. that the DFA D obtained from 
an NFA A recognises the same language as A) relies on a proof 
by induction on the length of the input string w, that 

õp ({q0}, w) is the set of all states q such that there exists a run 
of A on w from qo to q. 

We will cover this if we have extra time. 
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Another Generalisation 


What if we allow non-deterministic state changes that do not 
consume any input symbols? 


We label these silent moves with e (the empty string): 
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Another Generalisation 


What if we allow non-deterministic state changes that do not 
consume any input symbols? 


We label these silent moves with e (the empty string): 


Exercise: Is 001 accepted above? Can we express this as a 
DFA? 
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e-NFA to DFA 


The subset construction also applies to e-NFAs. 
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The subset construction also applies to e-NFAs. 


Define the e-closure E(q) of a state q as the set of all states 
reachable from q by silent moves. 
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satisfying: 

m qE E(q) 

m For any s € E(q), we also have ô(s,e) C E(q). 
We also extend this to sets, where E(S) = Uges E (q). 
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e-NFA to DFA 


The subset construction also applies to e-NFAs. 


Define the e-closure E(q) of a state q as the set of all states 
reachable from q by silent moves.That is, E(q) is the least set 
satisfying: 

m qE E(q) 

m For any s € E(q), we also have ô(s,e) C E(q). 
We also extend this to sets, where E(S) = Uges E (q). 


In our subset construction, everything is the same except that 
each subset (each state of our DFA) is e-closed: 


5p(S,a)=E (U 5a(s, a) 


ses 
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Summary 


DFAs, NFAs and e-NFAs all recognise the same class of 
languages, called the regular languages. They are equal in 
expressive power, although some representations (NFAs) are 
more compact than others (DFAs). 


10r this time, if we have time. 
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Summary 


DFAs, NFAs and e-NFAs all recognise the same class of 


languages, called the regular languages. They are equal in 
expressive power, although some representations (NFAs) are 
more compact than others (DFAs). 


Questions for next time 


m Are the regular languages closed under union? sequential 
composition? intersection? complement? 
(How would we prove this?) 

m What languages are not regular? 
(How would we prove this?) 


10r this time, if we have time. 


